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EAClvGROUrND 

Whether school curricula should be based on objectives specifying 
the behavioral changes expected of students in specific content areas has 
Deen a topic of continuing controversy. Advocjdtetj of beliavioial objectives 
(Bloom, 1956; Mager, 1952; Popham, 1970) claim that such a systematic 
approach to curriculum design leads to better instruction and maximizes 
student learning, while opponents (Atkin, 196B; Eisner, 1967a) argue that 
the precise statement of objectives leads to a mechanistic approach to 
instruction which har;pers the teacher's desire to innovate or pursue unan- 
ticipated outcomes, thus, curtailing student learning to some extent. 
One of the problems in this controversy over the use of behavioral objec- 
tives is that most advocates and opponents have formulated their position 
on the basis of a logical rather than empirical analysis of the instruc- 
tional process. This evaluation effort was planned in response to Eisner's 
statement, "that the contribution of educational objectives to the process 
of curriculum construction, teaching, and learning, is, at base, an empi- 
rical problem" (1967b, p. 277). 

OBJECTIVES OF THIS STUDY 

Recognizing the need to study the impact of educational objectives 
empirically, the objectives of this study were set as follows: 

1. To develop a model procedure for the empirical 
assessment of the contribution of behavioral 
objectives to teaching and learning; 

2. To use this procedure to empirically assess 
the contribution of behavioral objectives to 
teaching and learning within an actual school 
setting where objective based curricula are 
being used. 
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SETTING 

The model procedure for assessing the contribution of behavioral 
objectives to teaching and learning was developed within the context 
of a suburban Greater Boston middle school encompassing grades five 
tlirough eight. Within this setting, the systems approach to curriculum 
development was utilized to design objective-based curricula for all 
disciplines. 

Once the model procedure was developed it was applied in science, 
since the science curriculum had been implemented more extensively than 
the curricula in other disciplines. More specifically, this applica- 
tion focused on the unit of the science curriculum entitled, "Physics, 
Chemistry and Weather". The model procedure was applied to all classes 
at each grade level. The na^iber of classes at each grade level is re- 
ported in Table 1, 

TABLE 1 

The Number of Classes at Each Grade Level 
Participating in the Application of the Model Procedure 
For Assessing the Contribution of Objectives 
To Teaching and Learning 



Grade 


Kumi:>er of 
Classes 


5 


10 


6 


12 


7 


11 


8 


10 
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It is important to note that in developing the science curriculum 
for grades five through eight, a scope and sequence of the concepts and 
generalizations essential to the discipline was delineated. Then instruc- 
tional c'uides were developed for various units at each grade level on 
the bdcic cf thic cccpc ^nd ccTpjencc cf concepts find <^en^r'?i 1 i T^'^tions. 
Although a separate instructional guide was prepared for each grade 
level/ the procedure for implementing the science curriculum was such 
that teachers at all four grade levels taught the same basic unit at 
the same time. For example, when teaching the "Physics, Chemistry, and 
Weather" unit, the teachers in grades five through eight used different 
instructional guides at each grade level r but these instructional guides 
were all keyed to the scope and sequence of concepts and generalizations 
related to physics, chemistry and weather. The instructional guide at 
each grade level contained a listing of the objectives of the unit as 
well as the pre- and post tests for the unit based on parallel items. 

DEVELOPMENT OF A MODEL PROCEDURE FOR EMPIRICALLY 
ASSESSING THE CONTRIBUTION OF BEHAVIORAL 
OBJECTIVES TO TEACHING MID LEARNING 

The model procedure developed for assessing the contribution of 
objectives to teaching and learning is based upon two assumptions. First, 
if cognitive objectives have an impact upon teaching and learning, then 
teachers will teach toward the behaviors specified in -the objectives and 
students will master these behaviors. Secondly, the behaviors specified 
in the objectives of a curriculum as well as those behaviors taught by the 
teacher and mastered by the pupil can be classified according to the 
Taxonomy (Bloom, 1956). Once classified, these behaviors can be summarized 
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in the form of a coqnitive emphasis profile which indicates the percent 
of emphasis devoted to each taxonor.iic category. For example, the taxonomic 
classification of the objectives of a curriculum might produce the follow- 
ing; O i ;.r = \:v t."i cn : 

Kn.c o.lr • 50 objectives 50% 

Covapi.:-*. ri-'v • - 30 objectives 30% 

ApplicaLio^i - 6 objectives Converting to percents 6% 

Analysir, - 5 objectives : — — :— > 5% 

^ • p yields a cognitive 

Synthesis ~ 5 obiectives ^ . 5% 

_ , ^. >, ^. emphasis profile 

Evaluation - 4 obijectives r- t 

As noted above, a cognitive emphasis profile for the curriculum can be 
derived by converting the number of objectives in each taxonomic category 
to a percent. This conversion assumes that all objectives receive equal 
weight in the instructional process, regardless of their taxonomic level. 

Given the assumptions stated, the procedure for assessing the contri- 
bution of objectives to teaching and learning was developed within the 
context of the descriptive matrix of Stake's paradigm for the classifica- 
tion of evaluative data (Stake, 1967). This paradigm is illustrated in 
Figure 1. For the purpose of this study the columns and rows of the 
descriptive matrix can be viewed as follows: 

Intents - anticipated events or behaviors. 

Observations - events or behaviors which have 
oc cured , 

Antecedents - events or behaviors preceding 
the instructional process 

Transactions - events or behaviors dealing 

with the instructional process 

Outcomes - events or behaviors resulting 

from the instructional process. 

The model procedure developed in this study is based on six cognitive 

enphasis profiles corresponding to the six cells of the descriptive 



Q matrix of Stake's framework. 
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DEFINITION ADD DERIVATION OF THE COGNITIVE EMPHASIS PROFILES 

2. Profile based on the objectives of the curriculum 

When utilizing the systems approach to curriculum development 
(Banathy, 1968), objectives and tests are products of the curriculum 
development process presented to the teacher when the curriculum, is im- 
plemented. Since the objectives of a curriculum are developed prior to 
instruction and may be viewed as a statement of the need which the cur- 
riculum is to satisfy (Carter, 1969), it is appropriate to place objec- 
tives within the intended antecedents cell of the Stake paradigm. 

The cognitive emphasis profile based on objectives was derived by 
classifying objectives according to the Taxonomy of Intellectual Abilities 
(TIA) using the procedures developed by St:eele (1969) . 

Steele adapted the TIA from the Taxonomy (Bloom, 1956) in an effort 
to provide a classification system which could be used readily by teachers 
and administrators who have not received extensive training in the classi- 
fication of cognitive behaviors. 

The TIA system can be used to classify behavioral objectives and/or 
measurement items in terms of the following hierarchical listing of cogni- 
tive behaviors: 



I. 


Memory 


II. 


Translation 


III. 


Interpretation/Extrapolation 


IV. 


Application 


V. 


synthesis 


VI. 


Evaluation 


VII. 


Formal Analysis 
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An operational definition of each of the cognitive behaviors of the TIA 
is presented in the Appendix. Accordin9 to Steele (1970, p. 8), ''a 
reliability and validity study was conducted which demonstrated that 
the Taxonomy of Intellectual Abilities could be used reliably and with 

prCC2.3I.GIi by untrained teach^^rs ami auiVilitxSLidLui. In dUulLlOa/ 

Scrinivasan (1971) found that a translation of the TIA could be used 
reliably with an Indian population. 

For this study, the cognitive emphasis profiles based on the objec- 
tives of the science unit were derived by employing a panel of seven 
trained raters to classify the unit objectives at each grade level accord- 
ing to the TIA. Once the objectives at each grade level were classified, 
the cognitive emphasis profile based on objectives was derived by cal- 
culating the percent of objectives within each TIA category. A separate 
profile was developed for the "Physics, Chemistry, and Weather" unit at 
each grade level. 

To obtain an indication of the interrater reliability of the proce- 
dure used to rate the objectives, the intraclass correlation coefficient 
(Guilford, 1965) was calculated on the basis of the ratings obtained at 
each grade level- The reliabilities which were obtained are presented in 
Table 2. The high reliabilities presented in Table 2 indicate that there 
was considerable agreement among raters as to the taxonomic level of the 
behaviors reflected in the objectives at each grade level. 

In summary, the cognitive emphasis profile based on objectives is a 
measure of curriculum intent which conveys those behaviors which should 
be emphasized when the curriculuir is implemented in the school setting. 
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TABLE 2 

Reliahili ties By Grade Level Fcr The 
Classification cf the Unit Objectives 
According to the Taxonomy of 
Intellectual Abilities 



Grade 


Number of 

Objectives 

Classified 


Reliability 


5 


47 


.94 


6 


28 


.93 


7 


123 


.89 


8 


26 


.95 



2. Profile based on measurement items 

Tyler (1950) notes that if the tests developed for a curriculum do 
not measure the same behaviors as the objectives, then either the tests 
will not be used or the curriculum will be deflected from its original 
intents as stated in the objectives. Since teachers are usually required 
to use the tests provided when implementing an objective-based curriculum, 
it is safe to assume that instruction will be influenced by the behaviors 
measured by these tests. Consequently, the measurement items of these 
tests provide an indication of the behaviors v;hich teachers are encouraged 
to emphasize during instruction and can be classified according to the 
Stake paradigm as observed antecedents. 

The cognitive emphasis profile based on measurement items was derived 
by employing a panel of seven raters to classify the measurement item of 
the post tests for the "Physics, Chemistry, and Weather" unit according 
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to the Taxonomy of Intellectual Abilities. Then a profile based on 
measurement items for each grade level was obtained by calculating the 
percent of post test iteir.s within each TIA category. 

A measure of the reliability of these item ratings was obtained by 
calculating the intraclass correlation coefficient at each grade level. 
The reliabilities obtained are presented in Table 3. 

TABLE 3 

Reliabilities By Grade Level For The Classification 
Of The Measurement Items Of The Unit According 
To The Taxonomy of In^*^llectual Abilities 



Grade 


Number of 
Measurement 
Items Classified 


Reliability 


5 


38 


.58 


6 


24 


.78 


7 


27 


.76 


8 


27 


.86 



The reliabilities presented in Table 3 were lowered due to the nature 
of the measurement items being rated. At the fifth grade level 97.37 per- 
cent of the items measured memory types of outcomes and 2.63 percent of the 
items measured translation types of outcomes. Since at the fifth grade 
level there was a high concentration of measurement items v;hich the raters 
perceived as obviously measuring one specific outcome (i.e., memory), the 
variance among items was not largo compared to the variance among raters. 
In the ideal situation, it is expected that there would be considerably 
more variance among items than among raters. The small variance among 
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items as compared to the variance arr.ong raters tends to increase the eiaor 
variance and decrease the intraclass correlation coefficient. As grade 
level increased the measure^nnent items tended to measure a wider range of 
the behaviors described in the Taxonomy of Intellectual 7U:*ilities. Thus, 
tihe variance among items increased and the intraclass correlation coeffi- 
cient was not lowered as much. 

To summarize, the cognitive emphasis profile based on measurement 
items is a measure of curriculum practice which reflects those behaviors 
which will be enphasized when the curriculum is implemented in the school 
setting. 

3. Profile baaed on the behaviors the teacher planned 
to foster in the clasaroom. 

When a curriculum is implemented in the school setting, the teacher 
must utilize the curriculum materials which have been developed and his/ 
her personal resources to plan and present an instructional sequence suited 
to the needs of the class. Jackson (1968) in his book. Life In Classrooms , 
discusses the distinction between what teachers plan to do in the class- 
room and what they actually do in the classroom as follows: 

This distinction being made here between two aspects of 
the teacher's work is so fundamental and has so many implica- 
tions for educational matters that it deserves some kind of 
official recognition in the language used to describe the 
teaching process. The terms ' interactive ' and 'preactive ' 
might serve this purpose. IvTiat the teacher does vis-a-vis 
students would be called 'interactive teaching' and what he 
does at other times - in an empty classroom so to speak - 
could be called 'preactive teaching*. (Jackson, 1968, 
pp. 151-152) 

In light of Jackson's comments, the cognitive behaviors which the teacher 
plans to emphasize in the classroom can be viewed as the teacher's pro- 
active cognitive emphasis. The cognitive behaviors which the teacher 
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does emphasize in the classroom can be viewed as the teacher's inter- 
active cognitive emphasis. Since proactive cognitive emphasis is a 
measure of the behaviors the teacher plans to foster in the classroom, 
it can be classified according to the Stake paradigm as an intended trans- 
action. 

One approach to deriving a cognitive emphasis profile based on the 
behaviors the teacher planned to foster in the classroom would be to ask 
the teacher to complete the cognitive section of the Class Activities 
Questionnaire (Steele, 1969) on the basis of the behaviors he/she planned 
to emphasize in the classroom. The cognitive section of the Class 
Activities Questionnaire (CAQ) consists of fourteen short items which 
measure the seven cognitive dimens r ; of the Taxonomy of Intellectual 
Abilities. In the setting v.'her« .is model was being developed, the CAQ 
was not appropriate since it was designed for use at the secondary and 
post- secondary levels. To facilitate the derivation of the necessary 
cognitive ernphais profiles, the cognitive section of the CAQ was adapted 
for use at grades five through eight. The resultant adaptation of the 
CAQ was called the Science Activities Questionnaire (SAQ) . The items 
of the SAQ are presented by cognitive factor in Figure 2. A copy of the 
S.^Q is presented in Appendix. 

Each teacher participating in this study was asked to complete the 
Science Activities Questionnaire for each of his/her classes on the 
basis of the behaviors he/she planned to foster during instruction. The 
responses to the 5a; were weighted as follows: 



R esponse Weight 

Z t. TO nq i y Aq roe 4 
Agree 3 
Disagree 2 
Strongly Disagree 1 
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FIGURE 2 

Items of the Science Activities Questionnaire 

(SAQ) Pr:^Ge/ited by Cognitive Factor 

Factor I: Memory 

1. The students main job was to remember information. 

?• Memorizing information was the students main concern. 

Factor II: Translation 

6. Putting ideas into your own words was an important activity. 

13. It was important for the students to explain and to summarize 
the information which was presented. 

Factor III: Interpretation/Extrapolation 

4. Students were expected to come up with ideas or results that 
were suggested, but not clearly stated, in the information which 
was given. 

11. Students were expected to use the information which was given 
to figure out how things were related - that is# how things 
were the same or how things were different. 

Factor IV: Application 

3. It was important for students to use the knowledge and methods 
they had already learned to solve new problems by themselves. 
10. Students were expected to use the knowledge and the problem 
Golvinq skills they had learned in science to solve problems 
in real-life situations. 

Factor V: Synthesis 

8. Students were urged to produce something brand new by adding 
new information to what they already knew. 

14. Inventing # designing, composing, and creating were important 
activities. 

Factor VI: Evaluation 

2. It was important for students to decide whether their answers 
were right or wrong and to give the reasons for their decision. 

12. It was important for the students to decide whether the ideas 
presented were good or bad. 

Factor VII: Formal Analysis 

5. Drawing conclusions and examining the difference between facts 
and hypotheses were important activities. 

9. Using the scientific method to think through difficult problems 
was an important activity. 
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The teacher's score on each cognitive factor of the SAQ is the mean of 
the responses to the tuo items measuring tliat factor. Thus, the cogni- 
tive emphasis profile based on ti^e behaviors the teacher planned to 
foster in the classroorr*. is the teacher's profile of scores on the seven 
cognitive factors of the SAQ. 

Since paired items are used to measure the cognitive factors of the 
SAQ^ the percent agreement between responses to these paired items was 
used as an index of the reliability of the SAQ when administered to 
teachers. The reliabilities obtained are presented in Table 4. 

TABLE 4 

Reliability of the Science Activities Questionnaire 
In Terms of t^e Percent Agreement on the Paired Items 
Of Each Cognitive Factor 



Cognitive Factor 


Number of 
Teachers 
Re spend 1 ng 


Percent Agreement 


Memory 


44 


72.73 


Translation 


44 


9r.91 


Extrapolation/ 
Interpretation 


44 


90.91 


Application 


44 


93.18 


Synthesis 


30 


86.67 


Evaluation 


29 


89-66 


Formal Analysis 


31 


87.10 
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The reliability of the itens related to the memory factor was not 
as high as anticii.atua d'>- to ii^^i^ie teachers' concept of the memory pro- 
cess. Sone teachers felt that readinq a definition over and over until 
it could be recited fro^Ti menory could be called raemorization, but being 
able to recite a definition from memory because it was heard several 
times during class discussions was not memorization. These teachers 
responded to the items related to the memory factor by saying it was 
important to remember or recognize information but not important to 
memorize information. 

In summary, the cognitive emphasis profile based on the behaviors 
the teacher planned to foster during instruction may be viewed as a 
measure of instructional intent- 

4. Profile based on the student's perceptions of 
the behcviovs taught in the alaseroom. 

As previously discussed, the cognitive emphasis of the teachers 
within the instructional setting can be viewed in two ways - 1.) the 
cognitive behaviors which the teacher plans to emphasize (i.e., preactive 
cognitive emphasis), 2.) the cognitive behaviors which the teacher ac- 
tually emphasizes within tlie instructional setting (i.e., interactive 
cognitive emphasis). Since interactive cognitive emphasis is a measure 
of the behaviors actually fostered in the instructional setting, it can 
be classified according to the Stake paradign as an observed transaction. 

A logical source for information regarding the behaviors the teacher 
actually emphasized in the classroom is the students. A cognitive empha- 
sis profile based on students* perceptions of the behaviors taught in 
the classroom was obtained by administering the Science Activities 



Questionnaire to the class. VJhon usinq the SAQ with the class, students 
were ins true ttiJ to roi^i ohii to each itor, on tl^e basis of what the teacher 
had stressed during instrrtction. The item responses of students were 
weighted just like the item responses of the teacher, and the class score 
on each cognitive factor of the SAQ is the mean of the responses to the 
two itens measuring that factor. Thus, the cognitive emphasis profile 
based on students' perceptions of the behaviors taught in the classroom 
is the class profile of scores on the seven cognitive factors of the SAQ. 
A separate profile was developed in this manner for each class particip>at~ 
ing in this study. 

If the Science Activities ^questionnaire (SAQ) is a reliable instru- 
ment for the measurement of cognitive emphasis within a classroom, then 
it would be expected that there would be more agreement among the SAQ 
responses cf the students in the same class than among the SAQ responses 
of students from different classes. Horst (1949) has developed a formula 
which estimates the reliability of an instrument on the basis of the 
within class and between class variemces. According to the Horst formula 
an instrument is reliable if the within class variance is small as com- 
pared to the between class variance. The reliabilities of the cognitive 
factors of the SAQ obtained in this study as well as the reliabilities 
of the cognitive factors of the Class Activities Questionnaire (Steele, 
1971) are presented in Table 5. According to the data presented in 
Table 5, there are only minor differences between the overall reliabili- 
ties of the cognitive factors of the Science Activities Questionnaire and 
Steele's Class Activities Questionnaire. 
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TAbLE S 

ReliaL 11 i tit .:; by Co^jjiiti.ve Fr:ct(^r of the Clar^s 
Activities luustionnairo Deve.loped by Steele 
And the Science Activities questionnaire 
Used in This Study 



Cognitive Factor 


Class Activities 

Questionnaire 
(^J = 131 classes) 


Science Activities 

Questionnaire 
(N = 44 classes) 


Meinory 


.88 




.84 


Translation 


.65 




.83 


Interpretation/ 
Extrapolation 


.86 




.64 


Application 


.83 




.81 


Synthesis 


.89 




.79 


Evaluation 


.71 




.78 


Formal Analysis 


.78 




.88 


To s ij n;**^. r i Z 6 


the connitivp emnhf\c;ic; profilp 


based on 





perceptions of the behaviors taught can be viewed as a measure of the 
instructional practices of the teacher as perceived by the class. 

5. Profile based on the objectives taught by the teacher 

Although teachers usually focus on the objectives of the curriculum 
they are implementing, circumstances sometimes prevent them from direct- 
ing their instruction toward all of these objectives. In some cases the 
teacher may not cover a series of objectives since he/she feels the 
students have already mastered the objectives , or conversely, the objec- 
tives may be too complex given the ability level of the class. In other 
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instances, tn- teacher r.:ay find that there is not sufficient time to 
emphasize all th^i object ivfjr,, ii^ost school settings it is realistic 

to make the distinction between tl\o objectives of the curriculum and the 
objectives taught by the teacher* The objectives taught by the teacher 
can be viewed according to the Stake framework as intended outcomes since 
these are the behaviors which one can realistically expect the students 
to exhibit. 

A measure of cognitive emphasis based on the objectives taught was 
obtained by asking the teacher to keep a record of the objectives actually 
covered* Since all objectives were classified according to the Taxonomy 
of Intellectual Abilities in order to develop the cognitive emphasis 
profile based on the objectives of the curriculum, the cognitive emphasis 
profile based on the objectives taught by the teacher was derived by 
simply calculating the percent of objectives taught by the teacher 
within each TIA category. This profile reflects those behaviors which 
students are expected to exhibit due to instruction. 

6, Profile based on student achievement 

since student achievement is based on observed behavioral change, 
it can be classified as an observed outcome in terms of the Stake para- 
digm. In order to exhibit a positive behavioral change, the student 
must respond correctly to post test items v/hich were answered incorrectly 
on the pretest. Given that the measurement items of the post test have 
been already classified according to the Taxonomy of Intellectual 
Abilities to derive the cognitive emphasis profile based on measurement 
items, the cognitive emphasis profile based on student achievement was 
derived by 1.) keeping a tally of the students exhibiting a positive 
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behavioral ciiange on each iter:, of the post test, 2.) sunming the tallies for 
all items within the sarrie TIA catecjory to obtain the total number of behavioral 
changes within each TIA category and then, 3.) calculating the percent 
of behavioral changes within each TIA category. In suininary, the cogni- 
tive ernpl\asis profile based on student achievement is the profile of the 
percent of observed behavioral changes within each TIA category. 

Each of the cognitive emphasis profiles discussed is listed in 
Table 6 along with its classification recording to the descriptive matrix 
of Stake's paradigm and the general method used to derive the profile. 
These six cognitive emphasis profiles are the basic components of the model 
procedure for assessing the contribution of behavioral objectives to teach- 
ing and learning. These profiles provide a measure of cognitive intent 
and practice at the curricular, instructional and student outcome levels. 
The reliability iiilormation reported indicates that the cognitive empha- 
sis profiles can be derived within a school setting with a good degree of 
consistency. 

Now that the definition and derivation of the basic components of this 
model procedure have been discussed, it is appropriate to move on to a 
discussion of the relationships between these pj^of iles which can be exam- 
ined. 

RELATIONSHIPS BETWEEU COGNITIVE EMPHASIS PROFILES 
This model procedure for assessing the contribution of behavioral 
objectives to teaching an'-l learning focuses on an examination of the rela- 
tionships between the cognitive emphasis profiles which have been derived. 
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TABLE 6 

Suiryruary of the Cognitive EK\phasis Profiles 
CoiTiprising the Model Procedure for Assessing 
The Contribution of Behavioral Objectives 
To Teaching and Learning 



Profile 


Classification 
According to the 
Stake Paradign 


Method for Deriving 
1 The Profile 


Profile based on the objectives 
of the curriculum 


Intended 
Antecedent 


Classify Curriculum 
Objectives Accord- 
ing to the TIA 


Profile based on the measurement 
items of the post tests 


Observed 
Antecedent 


Classify Post Test 
Items According to 
the TIA 


Profile based on the behaviors 
the teacher planned to foster 
in the classroom 


Intended 
Transaction 


Teacher Response to 
SAQ 


Profile based on the students' 
perceptions of the behaviors 
taught in the classroom 


Observed 
Transaction 


Students* Response 
To SAQ 


Profile based on the objectives 
taught by the teacher 


Intended 
Outcome 


Classify Objectives 
Taught According to 
the TIA 


Profile based on student achieve- 
ment 


Observed 
Outcome 


Classify Behavioral 
Change According to 
the TIA 
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n¥Dre specifically, the relationships between the profiles based on objec- 
tives and each of the othor profiles. If objectives do contribute to 
teaching and learning, there should bo con^3id^rable siniilarity between 
the profile of cognitive behaviors based on the curriculum objectives 
and each of the other five profiles. 

The easiest v;ay to discuss the relationships examined in this model 
procedure is in terms of an exemplary classroom situation. Using the tech- 
niques discussed in the previous section, six cognitive emphasis profiles 
were developed for each science class within the middle school setting on 
the basis of the "Physics, Chemistry, and Weather" unit. In viewing one 
of these science classes through its cognitive emphasis profiles, the 
first question of interest is "should objectives contribute to teaching 
and learning within this classroom setting?" Insights into this question 
can be obtained by examining the relationship between the profile based 
oil the curriculum objectives and the profiles based on the measurement 
item?; of the post test. As previously mcr^tioncd, if the curriculuiu objec- 
tives and measurement items do not focus on the same beh^*^^yiors, then the 
instruction of the teacher will tend to be guided by the behaviors reflected 
in the measurement item, thus, reducing the probability that the curriculum 
objective will contribute to teaching and learning. 

The next question of importance is "Do objectives contribute to teach- 
ing within the classroom setting?" Insights into this question can be ob- 
tained by examining the following relationships between cognitive emphasis 
profiles; 

1. Profiles based on the curriculum objective vs. the 
profile based on the behaviors the teacher planned 
to foster in the classroom. 
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2. Profile based on t\u:: curriculum objectives vs. the 

profile Lascd on the cIl^.ss i j^rccpti-n of the behaviors 
taught in the cias.^:.rco:,; . 

2. Profile based on the curriculum objectives vs. the 
profile based on the objectives the teacher actually 
taught toward during instruction. 

For the relationships vh.ere there is similarity between the cognitive 
emphasis profiles being compared/ one can conclude that objectives are 
making a contribution to th-it component of the teaching process. 

The final question which can be addressed when applying this model 
procedure is "Do objectives contribute to learning within the classroom?" 
The answer to this question can be obtained by examining the relationship 
between the profile based on curriculum objectives and the profile based 
on student achievement. If the profile of beha^'iors achieved by the 
students is similar to the profile of behaviors reflected in the curricul^^ 
objectives, this is an indication that objectives are making a contributio 
to learning within the classroom setting. 

Although such an examination of the contribution of objectives to 
teaching and learning within one classroom setting is of interest, a 
topic of greater interest is the contribution of objectives to teaching 
and learning across classes within the school setting. This more 
generalizeable information can be obtained by conducting an examination 
of the contribution of objectives to teaching and learning within each 
of the classes within the school setting, summarizing these findings by 
grade level, and then inspecting such findings to see if any overall 
trends emerge. 

Given this model procedure and the cognitive emphasis profiles 
derived for the classes within the middle school setting, it is timely 
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to move on to a discussion of the actual analysis of these profiles and 
the general findiiKjs wnica emei-^.v.d rovj.irding the contribution of objec- 
tives to teaching and learning. 

APPLICATION OF THE MODEL PROCEDURE FOR ASSESSING THE 
CONTRIBUTIOU OF BErlAVlORAL OBvTECTIVES TO 
TEACHIN'G AND LEARrJiNG WITHIN AN 
OPERATIOMAL SCHOOL SETTING 

Analysis of the Relationships Betv;een Cognitive Emchasis Profiles 

Correlational techniques were used to examine the relationships 
between the cognitive emphasis profiles for each classroom. The Pearson 
product moment correlation coefficient (liays, 1963) was o?ed to assess 
the similarity between cognitive emphasis profiles v^here both profiles 
were expressed in the same units of measurement. Wnen the cognitive 
emphasis profiles being corpared were expressed in different units of 
measurement they were converted to ranks and their similarity determined 
using the Spearman rank ccrrelation coefficient (Siegel, 1956). The 
Spearman rank correlation coefficient was used where one profile was ex- 
pressed in terms of percentages and the other profile was expressed in 
terms of mean scores. Table 7 contains a listing of the relationships 
which were examined when applying the model procedure developed in this 
study to a classroom setting as well as the correlational technique em- 
ployed to asijess each relationship. 

It is important to discuss how the results of the analyses of the 
relationships between cognitive emphasis profiles were interpreted* First 
of all, this study should be viewed as an evaluation rather than an ex- 
perimental study. In his duscussion of the differences in methodology 
between evaluations and experimental studies, Guba (1965) cautions against 
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Correlation Techniques Used to Assess the Relationships 
Between the Cognitive Emphasis Profiles 
Derived in This Study 



Correlational 

Relationship Technique 

Used 



Profile based on curriculum objectives 
vs. Profile based on measurerr.ent items 



Pearson r 



Profile based on curriculum objectives 
vs. Profile based on behaviors the 
teacher planned to foster in the 
classroom 



Spearman r 



Profile based on curriculum objec- 
tives vs. Profile based on class per- 
ception of the behaviors taught in the 
classroom 



Epearman r 



Profile bas2d on curriculum objectives 
vs. Profile based on the objectives 
taught by the teachor 



Pearson r 



Profile based on curriculum objectives 
vs. Profile based on student achieve- 
ment 



Pearson r 



the use of the classical experimental design approach to the analysis and 
interpretation of the data collected in an evaluation study. Instead^ 
"special techniques of analysis and interpretation need to be developed 
which are especially suited to the data produced" (Cuba, 1965, p. 32). 

When applying correlational techniques to the comparison of cogni- 
tive emphasis profiles, the sample sizes were small (i.e., 7). Due to 
small sample sizes, the procedure used to determine whether a significant 



relationship exists between the two profiles must take into account the 
low power of the '-t.i t i:>Li' ai tv.rii's er.i^'loyed. For example, if the decision 
is made that there is a significant relationship between two profilei^, 
there should be as little doubt as possible that this decision is accurate. 
To maximize the validity of the results of this study, each relationship 
between cognitive emphasis profiles was evaluated in terms of the tradi- 
tional decision rules (Mcl^emar/ 1962) plus a second decision rule included 
to compensate for the low power of the statistical procedures used* The 
decision rules used are as follows: 

1. A relationship between cognitive emphasis p^rcfiles 
significant at the .05 level, was viewed as 
signiricemt for the purposes of this study. 

2. A relationship betv;een cognitive emphasis profiles 
not significant at the •OS level, but significant 
at the .10 level, was viewed as questionable for 
the purposes of this study. 

3. A relationship between cognitive emphasis profiles 
not significant at the •lO level, was viewed as 
not significant for the purposes of this study. 

This approach to interpreting the results of the correlational ana- 
lyses conducted is consistent with the intended use of the findings of 
this study. The findings of this evaluation based on the relationships 
tested should be viewed as indicators which ir^ust be substantiated through 
further replications before generalizations can be made. 

Study Findings 

Upon analyzing the relationships between the cognitive emphasis 
profiles based on the objectives of the curriculum and each of the other 
five cognitive emphasis profiles derived in this study, the results were 
summarized by grade level to identify overall trends. In order to identify 
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trends in these relationships across classes, the findings for classes at 
each grade level were v.vi^jhted as follows: 

Each si':Tnif icant relationship was assigned a weight 

of I . or . 

Each questionable relationship was assigned a weight 
of 0.50. 

i.\..:h nonsignificant relationship was assigned a weight 

These wci ;:.*- tlien suinmed for each relationship and a mean was cal- 

culated, Ti: resultant mean serves as an index of the strength of the 
trend for the relationship across classes at a particular grade level. 

The range of the index of strength corresponding to each of the 
general trenJo for the relationships of this study is presented in Table 8. 

TABLE 8 

Range of the Index of Strength Corresponding 
To Each of the General Trends for 
The Relationships of this Study 



C neral Trend Range of the 

Index of Strenqth 



Si-jnil leant: ( + ) 0.56 - 1.00 

iuo:^t:ionabl'- (?) 0.45 - 0.55 

Hot Sicjn : ■ L ' ant {-) 0.00 - 0.44 



- J .vf^l fi:; iings for the relationships examined is this 

ax r ; . :. ' rodol i:rocedure for assessing the contribution of 

Leh-i^ 1 ; X . . • ... teaching and learning are summarized by grade 

level in i: . m-jex of strength as well as the general 
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disposition of each relationship is provided. U'hen interpreting the index 
of strength it should be noted that the general trenu across classes moves 
toward significant as the index of strength increases and toward not 
significant as the index of strength decreases. 

On the basis of rhe results presented in Table 9. objectives made 
their greater contribution to teaching and learning at those grade levels 
where the objectives and measurement items focused on the same cognitive 
behaviors. At grades five and seven where the behaviors of the objec- 
tives were similar to those of the measurement items , teachers tended to 
teach toward these behaviors and students tended to master these behaviors. 
At grades six and eight where the objectives and measurement items did not 
focus on the same behaviors , teachers taught toward those behaviors reflect- 
ed in the curriculum objectives, but students did not master these beha- 
viors. 

Concerning the relationship between the behaviors reflected in the 
curriculum objectives and the behaviors emphasized during instruction, the 
results indicate that with the exception of grade eight, there is no 
consistency between the behaviors of the curriculum objectives and the 
behaviors which the teacher planned to emphasize in the classroom. At 
grade eight, there v;as no definite trend for this relationship. Relative 
to the relationship between the behaviors of the curriculum objectives 
and the behaviors which the class perceived as being emphasized during 
instruction, no definite trend emerged at any of the grade levels. This 
observation of no definite trend indicates that some classes perceived 
the behaviors of the curriculum objectives to be. emphasized in the' 
classroom while other classes did not. 



27 



TA3LE 9 

Cuinnary of th.o Class L-.-'vel Findings for the 
Relationships Betv^een Cognitive Emphasis 
Profiles r:xamined in this Study 



Trend for the Disposition of 



Relationship Examined 


The relationship by grad 


e level 




5 


6 


7 


8 


♦Profile based on curriculum 


+ 




+ 




objectives vs. profile based 




(0.00) 


(1.00) 


(0.00) 


(1*00) 


on measurement items. 


Profile based on curriculum 








? 


objectives vs. profile based 










on behaviors the teacher 


(0.30) 


(0.04) 


(0.23) 


(0.50) 


planned to foster in the 










classroom. 










Profile based on curriculum 


? 


? 


? 


? 


objectives vs. profile based 










on class perception of the 


(0.55) 


(0.54) 


(0.50) 


(0.50) 


behaviors taught in the 










classroom. 










♦Profile based on curriculum 


+ 


+ 


+ 


+ 


objectives vs. profile based 










on the objectives taught by 


(0.90) 


(1.00) 


(1.00) 


(1.00) 


the teacher. 










♦Profile based on curriculum 


+ 




+ 




objectives vs* profile based 










on student achievement. 


(l.OC) 


(0.00) 


(1.00) 


(0.00) 



♦See discussion of tindmgs, page 
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Discussion of the Finding s 

An extended Jiscui:bion of ^^'JS'? findiiujs and hov; they may have been 
biased by the procedures utilized to derive the cognitive emphasis profiles 
is beyond the scope of this presentation. Persons interested in such a 
discussion may refer to Iwanicki (1972). 

It should be noted at this time that in reassessing this model pro- 
cedure v/hich has been developed , the findings reported concerning the rela- 
tionships in Table 9 which have been asterisked (*) tend to be unbiased by 
the procedures employed and representative of the contribution of objec- 
tives to teaching and learning in the setting where the model was applied. 
The other relationships involved cognitive emphasis profiles based on 
ponses to the Science Activities Questionnaire which posed some serious 
validity problems. 



CONCLUSION 

Through this study, a model procedure has been developed which can be 
used in an operational school setting to empirically assess the contribu- 
tion of cognitive behavioral objectives to teaching and learning. It 
is the intent of the authors that this rationale will be continuously refined 
and replicated in other school settings. Although the actual findings of 
this study cannot be generalized to other settings, the empirical evidence 
accumulated through subsequent replications will lead to the formulation of 
generalizations about the contribution of cognitive behavioral objectives 
to teaching and learning. This approach to establishing the generalizeabi- 
lity of a phenomenon is best described as follows: 

Because of the probabalistic nature of field data, and 
the impressionistic way that these are gathered, constant 
replication and recycling are necessary to build confidence 
in conclusions. This tactic might be called the tactic of 
Q accumulative evidence. (Cuba, 1965, p. 26) 

ERIC 
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Operational Definitions of Each of the Categories of the TIA 
as Adapted frora Bloon's Taxoiiony by Steele (1970, p. 7). 



Recall, recognition, bringing tc mind of any kind of informa- 
tion. Some alteration of the material nay be required, but 
this is a minor part of the task. Memory involves the ability 
to reproduce or recognize information as it was presented. 



II. 

Changing information into a different symbolic form to express 
the same idea, r^uch as the use of paraphrasing, pictures, 
graphs, summaries, outlin<3S, vr statements in technical or 
layman's languaqe. It also includes the use of metaphor, sym- 
bolism, and other non-literal statements. Translation involves 
the ability to comprehend information including recasting or 
altering it in various ways. 



Ill . IIJTEPvP'U^T^TIOU/EXTRAPOLATION 

Discovering and exuloriag the interrelationships among ideas 
(on a common-sense level). Comparing, contrasting, and 
explaining information based on the new viev; the perceived 
relationships provide. The task may require going beyond the 
given data in making inference??, predicting trends, and 
determining implications and consequences. Interpretation 
involves tho» ability to extend and manipulate information to 
clarify relationships suggested by the data or to project 
trends based on patterns apparent in the data. 



IV. rv'PLiCATio:] 

utilising abstractions (generalizations , rules , skills) in 
concrete situations. Selecting and applying rules or methods 
to solve a specific prcblem, usually with a minimum of direc- 
tion or prompting as to which abstractions apply or how to 
use them. This kind of task gives practice in the independent 
use of knov; ledge and skills, requiring the identification of 
the issue as well as selection and use of the correct abstrac- 
tions to solve problems in practical settings. Application 
involves the cibility to select the methods or generalizations 
called for by specific problem situations and perform the 
operations required to solve the problem. 
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V. 



RecoTTibining parts of previous experience v;ith nev/ material 
into a nev7 inteorratod v/hole^ p^tteru^ or structure not clearly 
there before. Synthesis implies a new product requiring 
original y creative thinking. This can take the forra of a 
unique comiunication involving skill in v/riting or speaking; 
a proposed set of operations^ such as ways of testing hypotheses 
or developing an effective plan to solve a coinplex problem; 
or the derivation of abstract relations^ as in making generaliza 
tions or r^atheraatical discoveries. Synthesis involves the 
ability to generate nevr ideas and solutions; inventing , design- 
ing, composing, creating. 



Clarifyina and using a standard of appraisal in making judgments 
about the value of materials or methods for given purposes. 
In making judgments of good or bad, right or wrong, the stan- 
dards or criteria u sed should be made explicit . This category 
forms a ma^cr link with the affective domain where values, 
liking, and enjoying are central processes. Evaluation is 
always somewhat subjective because either the standard cannot 
be proven to be correct or the idea to be judged cannot be 
proven to violate or illustrate the standard. Evaluation in- 
volves the ability to develop and apply a set of standards for 
judging worth, and to support the judgments v/ith a justification 
or rationale based on the criteria used. 



Conducting a searching inquiry into the true nature of inter- 
relationships anc^ testing t!ie validity of arguments according 
to appropriate rules of reasoning, with conscious knowledge of 
the intellectual processes being performed. Analysis includes 
the ability to recognize unstated assumptions, distinguish 
facts from hypotheses and normative statements, recognize 
conclusions and purposes of the material, and check consistency 
and relationships. Formal "analysis involves the ability to 
consciously apply appropriate rules of reasoning in testing the 
validity of statements, arauments, aiid conclusions. 



VI. 



EVALUATION 



VII. 



FOR?"LAL AIJALYSIS 



SCIENCE ACTIVITIES QUESTIONNMRE 



Science Teacher's Name 
Science Section NTjn±)er 



Directions 

I^ch sentence in the Science Activities Questionnaire will be read to you and explained 

by your teacher. After the sentence is explained, decide hew well the sentence 

describes what was done in science during the last unit. When deciding, think of vy^at 

was done in science both in class and outside of class. 

It you are ver^^ sure that the sentence describes what was done 
m science, circle the wcrds - STRONGLY AGREE. 

If you think that the sentence describes what was done 
in science, circle the word - AGREE. 

If you think that the sentence DOES NOT describe \^^at was done 
in science, circle the word - DISAGREE. 

If you are very sure that the sentence DOES NOT descrilDe what was done 
in science, circle the words - STRONGLY DISAGREE. 

Please look at the examples telow. 

Example A 

- M 

The students' main job was to remeni^er information. 



^RDNGLY AGKE^ AGREE DISAGREE STRCM^LY DISAGREE 

In Example A, the student answering was very sure that the sentence described 
what was done in science during tJie last unit, so he circled - STRONGLY AGREE. 

E xample B 

Inventing, designing, oomposing, and creating are important activities. 



^ DISAGR^J 



STRONGLY A3REE AGREE ( DISAGREE ] STHONGLY DISAGREE 



In Example B, the student answering thought that the sentence DID NOT describe 
what was done in science during the last unit, so he circled - DISAGREE. 

ROTember to base your answer on hc^ well each sentence describes what your science 

teacner had you do in science class and outside ot science class during the last unit. 

Circle only one answer for each sentence. Use pencil only. If you change an answer, 

erase the first answer completely. PLEASE DO NOT SELECT AN ANSWER UOTIL THE SEOTENCE 
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^ READ TO YOU AND EXPLAINED BY YOUR TEACHER. 



1. 'I'ho stu..io:its' rum job was Lo roiTieinLx}r mfoiTiution. 

Si'RONGLY AGREE aGREF: DISAGREE STRONGLY DISAGRI-^E 

2. It was inportant for students to decide whether their answers were right or 
v.Tonq and to give the reasons for their decision. 

AGREE AGi^ DISAGi^ STRONGLY DISAGREE 

J. It was LnifX)rtant for students to use the kncwledge and inethods they had already 
1 0:1 mod to solve new problems by thonselves, 

STIONGLY AGRliE AGREE DISAGREE STRONGLY DISAGREE 

4. Students were expected to cone up with ideas or results that w^ere suggested, but 
not cle.orly stated, in the information wnich was given. 

STRONGLY AGREE AGREE DISAGREE STRONGLY DISAGREE 

5. Drawing conclusions and examining the differences between facts and hypotheses 
wr-re important activities. 

STRONGLY AGRFJi AGREE DISAGREE STRONGLY DISAGREE 

6. Tatting ideas into your c^n words was an important activity. 

STROrvIGLY AGREE AGREE DISAGREE STRONGLY DISAGREE 

7. Memorizing information was the students mam concern. 

STRONGLY /^GREE AGREE DISAGREE STRO:)GLY DISAGREE 

8. Students were urged to produce sOTiething brand new by adding new information 
to what they already knew. 

STRONGLY AGREE AGREE DISAGREE STRONGLY DISAGREE 

9. Using the scientific method to think through difficult problems was an 
Liijjortant activity. 

STROrCLY AGREE AGREE DISAGREE STRONGLY DISAGREE 

10. Stucients were expected to use the knowledge and the problen solving skills they 
nad learned in science to solve problons in real-life situations. 

SITONGLY AGREE AGREE DISAGREE STRONGLY DISAGREE 

11. Students were expected to use the information which was given to figure out how 
tilings were related - that is, how tnings were the same or how things we^*9 different. 

STRONGLY AGREE AGREE DISAGREE STRONGLY DISAGREE 

12. it was important for the students to decide whether the ideas presented were good 
or bad. 

STOONGLY AGREE AGREE DISAGREE STRONGLY DISAGREE 

13. It was important for the stuaents to explain ana to sunmarize the information 
which -was presented. 

STK):r,LY AGREE AGREE DlfAGREE Si'RONGLY DISAGREE 

14. Invonting, designing, ccmposing, and creating were important activities. 

STRX^GLY AGPEE AGREE DISAGREE STRONGLY DISAGI^ 
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